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Abstract 

Optimal link adaption to the scattering function of wide sense stationary uncorrelated scattering 
(WSSUS) mobile communication channels is still an unsolved problem despite its importance for 
next-generation system design. In multicarrier transmission such link adaption is performed by 
pulse shaping, i.e. by properly adjusting the transmit and receive filters. For example pulse shaped 
Offset-QAM systems have been recently shown to have superior performance over standard cyclic 
prefix OFDM (while operating at higher spectral efficiency). In this paper we establish a general 
mathematical framework for joint transmitter and receiver pulse shape optimization for so-called 
Weyl-Heisenberg or Gabor signaling with respect to the scattering function of the WSSUS channel. 
In our framework the pulse shape optimization problem is translated to an optimization problem 
over trace class operators which in turn is related to fidelity optimization in quantum information 
processing. By convexity relaxation the problem is shown to be equivalent to a convex constraint 
quasi-convex maximization problem thereby revealing the non-convex nature of the overall WSSUS 
pulse design problem. We present several iterative algorithms for optimization providing applicable 
results even for large-scale problem constellations. We show that with transmitter-side knowledge 
of the channel statistics a gain of 3 — 6dB in SINR can be expected. 

Index Terms 

OFDM, OQAM, IOTA, frames, Wilson basis, Gabor signaling, WSSUS, Weyl-Heisenberg 
group 



I. Introduction 

It is well known that channel infoimation at the transmitter increases link capacity. How- 
ever, since future mobile communication is expected to operate in fast varying channels, it is 
not realistic to assume perfect channel knowledge. On the other hand statistical information 
can be used which does not change in the rapid manner as the channel itself. In multicarrier 
communications this can be employed for the design of transmitter and receiver pulse shapes. 
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Unfortunately the problem of optimal signaling in this context is still an unsolved problem. 
Orthogonal Frequency Division Multiplexing (OFDM) has the capability to resolve the inher- 
ent structure of time-invariant (or slowly fading) channels, i.e. converts the frequency-selective 
channel into flat fading channels on the corresponding subcarriers. From mathematical point 
of view the joint transmitter and receiver signaling (that includes an appropriate cyclic prefix) 
diagonalizes a complete class of linear time-invariant channels. Apart from the cyclic prefix, 
which implies bandwidth and power efficiency loss the classical OFDM approach seems 
to be an efficient setup for the time-invariant case. But this does not hold anymore if 
we consider additional time-variant distortions caused by the mobile channel or non-ideal 
radio frontends [1]. The conventional OFDM scheme can be extended in several ways to 
match the requirements for more mobility and increased bandwidth efficiency. In particular 
an approach based on Offset-QAM (OQAM) [2] in conjunction with Gaussian-like pulse 
shapes (OQAM/IOTA) [3], also incorporated in the 3GPP OFDM study item [4], [5], reflects 
a promising new direction. Due to the enhancement of the physical layer an improvement 
of the overall system performance is expected while the air interface is still very similar to 
OFDM. Thus, it makes sense to consider a more general signaling, namely Weyl-Heisenberg 
signaling, and assess the problem of optimal pulse shapes for a given second order statistics 
of the time-variant scattering environment. 

The paper is organized as follows. In the first part we formulate, what is mainly known 
under the name Weyl-Heisenberg (or Gabor) signaling. Then we will review in this context 
cyclic prefix based OFDM (cp-OFDM) and pulse shaped OQAM. In the second part of the 
paper we present the principles of WSSUS pulse adaption for Weyl-Heisenberg signaling. 
We will establish the main optimization functional and show several design strategies for its 
maximization. Then we give a more abstract formulation and identify pulse optimization 
as convex-constraint (quasi-) convex maximization problems. In the third part we then 
explicitely work out the optimization strategies and algorithms. Finally the performance of 
the iterative algorithms is evaluated. 



Conventional OFDM and pulse shaped OQAM can be jointly formulated within the concept 
of generalized multicarrier schemes, which means that some kind of time-frequency multi- 
plexing will be performed. To avoid cumbersome notation we will adopt a two-dimensional 
index notation n = (ni, 77-2) G for time-frequency slots n. In our framework the baseband 
transmit signal is 



II. System Model 




(1) 
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where (5'^7)(t) = e*^'^'^^*7(t — /ii) (i is the imaginary unit and /i = (/ii,yU2)) is a time- 

def 

frequency shifted version of the transmit pulse 7, i.e. 7„ = S\n 7 is shifted according to a 
lattice AZ^ (A denotes its 2 x 2 real generator matrix). The indices n = (ni, 722) range over the 
doubly-countable set X C Z^, referring to the data burst to be transmitted. Due to this lattice 
structure in the time-frequency plane (or phase space), this setup is sometimes called Gabor 
signaling. Moreover, because the time-frequency shift operators (or phase space displacement 
operators) are unitary representations of the Weyl-Heisenberg group (see for example [6], 
[7]) on £2(1^) this also known as Weyl-Heisenberg signaling. In practice A is often restricted 
to be diagonal, i.e. A = diag(T, F). However, Gabor based multicarrier transmission can 
generalized to other lattices as well [8]. The time-frequency sampling density is related to 
the bandwidth efficiency (in complex symbols) of the signaling, i.e. e = |detA^^|, which 
gives e = {TPy^ for A = diag(r, F). 

The coefficients x„ are the complex data symbols at time instant ni and subcarrier index 
77-2 with the property E{xx*} = I (from now on ^ always denotes complex conjugate and 
■* means conjugate transpose), where x = (. . . ,Xn, . . . )^. We will denote the linear time- 
variant channel by Ti and the additive white Gaussian noise process (AWGN) by n{t). The 
received signal is then 

rit) = ins){t) + n{t) = I i:{^j){S,,s){t)d^i + n{t) (2) 

with S : — i> C being a realization of the (causal) channels spreading function with 
finite support. We used here the notion of the wide-sense stationary uncorrelated scattering 
(WSSUS) channel [9] and its decomposition into time-frequency shifts. In the WSSUS 
assumption the channel is characterized by the second order statistics of 5](-), i.e. the 
scattering function C : ^ M+ 

E{S(/i)S(;;7)} = C(/i)5(/i-/i') (3) 

Moreover we assume E{S(yu)} = 0. Without loss of generality we use ||C||i = 1, which 
means that the channel has no overall path loss. To obtain the data symbol Xm the receiver 
projects on gm = SAm9 with m G X, i.e. 

Xm = {9m,r) = {SAm9,r) = J e~'^^^''^^''g{t-{Am),)r{t)dt (4) 

By introducing the elements 

Hm.n= {g7n,T-Cln) = / T,{fl) {Qm, S ^'^n) dfi (5) 

of the channel matrix H E C^^^, the multicarrier transmission can be formulated as the 
linear equation x = Hx + n, where n = (. . . , {gm, n), . . . )^ is the vector of the projected 
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noise having variance cr^ := Fin{\{gm,n)\'^} per component. If we assume that the receiver 
has perfect channel knowledge (given by S) "one-tap" (zero forcing) equalization would be 
of the form = H:^^Xm (or alternatively MMSE equalization if cr^ is known), where 

H^^^ = [ S(/i)e*2-(w(A'")2-/^2(^"^)i)(^,5^7)d/i= / S(/i)e*2-(/^i(^'^)2-/^2(^™)i)A3^(/i)c//x 

(6) 

Here Ag^{fi) = {g, is the well known cross ambiguity function of g and 7. 

We adopt the following ^2-normalization of the pulses. The normalization of g will 
have no effect on the later used system performance measures. The normalization of 7 
is typically determined by some transmit power constraint and will scale later only the 
noise variance cr^ cr^/ll7ll2- Thus we assume g and 7 to be normalized to one, i.e. 
Il^lh = II7II2 = 1- Furthermore we will not force orthogonality, like orthogonal transmit 
signaling ((7m,7n) = ^^n), orthogonality at the receiver ({gm,gn) = 5mn) or biorthogonality 
between transmitter and receiver ({gm,ln) ~ <^mn)- But note that, advanced equalization 
techniques (not considered in this paper) like interference cancellation will suffer from noise- 
enhancement and noise correlation introduced by non-orthogonal receivers. 

A. Complex Schemes 

In this approach full complex data symbols are transmitted according to ©. Depending 
on the lattice density (e < 1) this includes redundancy. In the sense of biorthogonality it is 
then desirable to achieve {gm, 'H'jn} ~ ^mn for a particular class of channels Ti. For example 
the classical OFDM system exploiting a cyclic prefix (cp-OFDM) is obtained by assuming 
a lattice generated by A = diag(T, F) and setting 7 to the rectangular pulse 

7(^) = -i=i==Xl-T.,,T^]{t) (7) 

The function X[-Tcp,Tu] is the characteristic function of the interval [—Tcp,Tu], where T„ 
denotes the length of the useful part of the signal and Tcp the length of the cyclic prefix 
(^ 10%TJ, hence the OFDM overall symbol period is T = T^ + T^p. The OFDM subcarrier 
spacing is F = 1/T„. At the OFDM receiver the rectangular pulse g{t) = -^Xlo,Tu]{'t) is 
used which removes the cyclic prefix. The bandwidth efficiency of this signaling is given as 
e = (TPy^ = Tu/{Tu + Tcp) < 1. It can be easily verified that A<,^((r + miT,m2F)) = 
\/^Sm,o if < r < Tcp (or see [1] for the full formula), i.e. 

Hm,n = {gm,'H-in) = \/eh{m2F)5m,n (8) 

holds for all channel realization as long as the causal scattering function fulfills S/j = and 
Td < Tcp, where Td (Bd) is its maximal delay (one-sided Doppler) support. h{f) denotes 
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the Fourier transform of the impulse response h{r) = S((r, 0)) that corresponds to the 
time-invariant channel. Therefore cp-OFDM is a powerful signaling, which diagonalizes time- 
invariant channels, but at the cost of signal power (the redundancy is not used) and bandwidth 
efficiency. 

Eq. dH]) does not hold anymore if the channels are doubly-dispersive, as for example 
modeled by the WSSUS assumptions. If considering other pulse shapes independent of a 
particular realization Ti it is also not possible achieve a relation similar to ([8]), which will 
be explained later on. So it remains to achieve at least {gm, 7n) ~ <^mri at nearly optimal 
bandwidth efficiency e ~ 1. But one of the deeper results in Gabor theory, namely the 
Balian-Low Theorem (see for example [10]), states that (bi-)orthogonal pulses at e ^ 1 must 
have bad time-frequency localization properties (at e = 1 diverging localization). Indeed, 
in discrete implementation the localization of orthogonalized Gaussians for e < 1 and 
"tighten" Gaussians for e > 1 peaks at the critical density e = 1 so that pulse shaping 
is mainly prohibited for band efficient complex schemes if still (bi-) orthogonality is desired. 
Nevertheless in contrast to cp-OFDM it is via pulse shaping (for e < 1) still possible to make 
use of the redundancy. 

B. Real Schemes 

For those schemes an inner product Re{(-, ■)} is considered, which is realized by OQAM 
based modulation for OFDM (also known as OQAM/OFDM) [2]. It is obtained in ^ and 
with a lattice generated by A = diag(T, F) having | det A| = 1/2. Before modulation the 
mapping Xn = has to be applied where G M is the real-valued information 

to transmit. After demodulation x^^ = Re{i^'^Xm} is performed. Moreover, the pulses 
((7,7) have to be real. Thus, formally the transmission of the real information vector = 
(. . . , xj^, . . . )^ can be written as = H^x^ + where the real channel matrix elements 
are: 

Hl„ = Re{z'^-'"i/,„,„} = Re{z"-'"(^7„, 7^7^)} (9) 

and "real-part" noise components are = Re{i~^ {g^n, n)} . Note that there exists no 
such relation for OQAM based multicarrier transmission equivalent to dH]) for cp-OFDM. 
Hence, also in time-invariant channels there will be ICI. But in the absence of a channel, 
biorthogonality of the form ^{{gm,ln)} = ^m,n can be achieved. Furthermore it is known 
that the design of orthogonal OQAM based multicarrier transmission is equivalent to the 
design of orthogonal Wilson bases [11]. Because the system operates with real information 

'We use here the notation i" = Furthermore other phase mappings are possible, like j"i+"2+2nin2 
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at e = 2 the effective efficiency is 1, but in the view of pulse shaping it is not affected by 
the Balian-Low theorem. It is known that the construction of orthogonal Wilson bases is 
equivalent to the design of tight frames having redundancy two (which will be explained 
later on in the paper) [12]. It will turn out that this equivalence holds also for the WSSUS 
pulse shaping problem considered in this paper if assuming some additional symmetry for 
the noise and the spreading function of the channel. Finally, extensions of classical Wilson 
bases to non-separable lattices are studied in [13]. 

III. WSSUS Pulse Design 

In the first part of this section we will collect, what in principle is known in WSSUS 
pulse shaping theory. The result are partially contained in [14], [8], [1], [15]. We begin 
by establishing a cost function which characterizes the averaged performance of uncoded 
multicarrier transmission over a whole ensemble of WSSUS channels. Even though we 
consider solely uncoded transmission, our results will give insights into the coded performance 
as well. However, an overall optimization of the coded performance is beyond of the scope 
of this paper. In the aim of maximizing the performance (the cost) we will show how various 
design rules on WSSUS pulse shaping occur as steps in this optimization problem. We will 
explicitely identify the stage at which this argumentation will have a gap which will be 
filled by our algorithms presented later on. In the second part of this section (in IIII-BI) we 
will establish a new analytical framework which better highlights the underlying algebraic 
structure. In particular it will turn out that this is important to understand the appropriate 
optimization strategies presented in the next section. 

A. The WSSUS Pulse Design Problem 

In multicarrier transmission most commonly one-tap equalization per time-frequency slot 
is considered, hence it is naturally to require a (the channel gain of the lattice point m E T) 
to be maximal and the interference power b from all other lattice points to be minimal as 
possible, where 

a = \Hm,m\'^ and b = \Hm,n\'^ (10) 

for complex schemes. For real schemes Hm,n has to be replaced by from 

This addresses the concept of pulse shaping, hence to find good pulses {g, 7} such that its 
averaged cross ambiguity yields maximum channel gain and minimum interference power. A 
comprehensive framework for the optimization of redundant precoders and equalizers with 
respect to instantaneous time-invariant channel realizations (assumed to be known at the 
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transmitter) is given in [16], [17]. However in certain scenarios it is much more realistic to 
adapt the pulses only to the second order statistics, given by C{fi) and not to a particular 
realization S(/i). Hence, instead we define the (long term) averaged signal-to-interference- 
and-noise ratio SINR(5f, 7, A) as 

S.NRfe.7.A)*.'^5^ (11) 

where the averaged channel gain and the averaged interference power are given for complex 
schemes as 

¥.u{a} = ¥.u{\HmM= f \{g,S^-1)\'C{^^)d^^ 

r (12) 

nGX\{m} neX\{0} '^^'^ 

Note, that both are independent of m. Thus in average all lattice points have the same SINR. 
Eq. ([TT]) will hold also for the real schemes from Sec lII-BI if we assume further that the 
spreading function for each ji and the noise process n{t) for each t are circular- 

symmetric (real and imaginary parts have same variances and are uncorrelated). We have 
then for each m, n: 

E^{|i7^,j2} = l(E^{|i7^,„p + Re{<J}) = \^^{\H^,,r.?} (13) 

because for circular-symmetric S follows E->^{iJ^„} = 0. Similarly we get for each m: 
E„{|n^p} = |(T^. Thus, for complex and real schemes the optimal time-frequency signaling 
{g, 7, A} in terms of SINR is now given as the solution of problem 

max ^ J: (14) 

(5,7,A) + E-h{&} 

Additional to the norm constraint applied on the pulses there has to be a bandwidth efficiency 
constraint on A in the sense of | det A| = e"^ = const. This stands in contrast to capacity 
(instead of SINR) optimization with respect to scattering knowledge at the transmitter. The 
capacity optimization problem itself is unsolved and it is unclear that it could make sense or 
not to tolerate slightly increased interference but operate at higher effective rate. 

The design problem (114)) in this general constellation is not yet well studied because of 
its complex structure. Most studies in this field are limited to separated optimizations of 
either E->^{a} or E^^j^} where the lattice structure A is assumed to be fixed. A comparison 
between different lattices is given in [8]. But clearly there must be some connection between 
E->^{a} and E->^{6}. For an orthogonal basis this is apparent where the general case can be 
established by frame theory [18], [19], [20]. Thus if we define for arbitrary (not necessarily 
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diagonal) A a Gabor set as Q{^,K,7?) := {S'A„7|n G Z^} we can associate to it a positive 
semidefinite operator 5^ a as follows 

(^,,A/)(t):= 5^ {S,^J){S,^m (15) 
If there are constants Aj > and < oo such that for all / G £2(R) holds 

AII/ll2<(/,5,,A/)< 5,11/11^ (16) 

^(7,A,Z^) is said to be a frame for £2(1R)- In this case S^ a is called the Gabor frame 
operator associated to 7 and Al?. If the upper bound in (fT6l) holds, ^(7, A,Z^) is called a 
Bessel sequence and the optimal (minimal) is called its Bessel bound [19]. Clearly B^ 
is the operator norm of 5^ a induced by ||-||2, i.e. B^ is the spectral radius B^ = p(S'^ a)- 
If Aj = B^ the frame is called tight and the frame operator is then a scaled identity, i.e. 
S^^A = B^I. In this case and if furthermore 7 is normalized the Bessel bound represents 
the redundancy of {{SAnl^f)]^ ^ Z^} for a given function /. The redundancy is related 
to AZ^ only, hence tight frames minimize the Bessel bound for fixed A. And in this sense 
tight frames can be seen as generalization of orthonormal bases for which then would hold 
B^ = 1. For Gabor frames (or Weyl-Heisenberg frames) one has further 

1 < min B^ = I det A"M = e (17) 

Il7l|2=l 

For e < 1 the set ^(7,A,Z^) can not establish a frame, i.e. A^ = 0. But it can establish a 
Riesz basis for its span where the minimal Bessel bound is attained for the orthogonal case. 
But the latter can be formulated within the frame techniques too. One important result from 
Gabor theory is the Ron-Shen duality [21] between lattices generated by A and its adjoint 
lattice generated with A° = det(A)^^A. The Gabor set ^(7, A, Z^) establishes a frame (tight 
frame) iff the Gabor set ^(7, A°, Z^) is a Riesz basis (ONBg basis) for its span. 
Assuming from now on that ^(7, A, I?) is a Bessel sequence we get immediately from (fT6l) 

En{a} + En{b} = J2^n{\Ho,n\'} < J2 ^'H{\Ho,n\'} 

= En{{9,nS^,An*g)} < B,En{\\n* gg} 
or equivalently E->^{6} < B^ — E->^{a} for ||C||i = 1 and ||5f||2 = 1 (Ti* is the adjoint 
operator of Ti with respect to standard inner product). Similarly follows from ([T3]) that 5^/2 
is the upper bound for real schemes in the last equation. This in turn gives for both - real 
and complex schemes - the lower bound 

SlNR{g, 7, A = .^"g > ,^r, I SI (19) 
+ E-Hib} + B^ - En{a} 

^ONB=Orthonormal basis 
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used already in [1]. If ^(7, A, 1?) is a frame there is a similar upper bound given with A^. 
Equality is achieved if X = and ^(7, A,Z^) already establishes a tight frame. Note that 
depends on 7 and AZ^ but is independent of the channel where E->^{a} depends on 
{7,(7} and on the channel, but is independent of the lattice again. A joint maximization of 
the lower bound would have similar complexity as the original problem, whereas separate 
optimizations of and E->^{a} seems to be much simpler. Eq. (fT9] ) motivates a design 
rule that optimizes the pulse with respect to the channel first and performs corrections with 
respect to a particular lattice afterwards. Thus we propose the following two-step procedure: 
1) Step one {Gain optimization): In the first step the maximization of the averaged channel 
gain E^^ja} is considered, which is 

En{a}= f \{g,S,^)\^C{fi)dfi= [ | A,^(/i)|2C(/i)d/i < ||C||i||(7||^||7||^ = 1 (20) 

In this context (l20l) was first introduced in [22] respectively [14], but similar optimization 
problems already occurred in radar literature much earlier. In particular for the elliptical 
symmetry of C(-) Hermite functions establish local extremal points as found in [14]. If C(-) 
is a two-dimensional Gaussian the optimum is achieved only using Gaussian pulses for g and 
7 matched in spread and offset to C(-) (see [23]). There is a close relation to the channel 
fidelity in quantum information processing which will become more clear in Section IIII-BI In 
IIV-BI we will establish the maximization of (l20l) as global-type optimization problem closely 
related to bilinear programming. However, already in [24] we have proposed the following 
lower bound 

En{a} > \{g, 5,C(/i)d/i) 7)!' = l(^?,/:7)r (21) 

which admits a simple direct solution given as the maximizing eigenfunctions of C*C re- 
spectively CC*. Furthermore the lower bound is analytically studied in [25]. 
Pulse Scaling: The maximization of (|20|) is still a difficult task, numerically and analytically. 
However, it is possible to obtain a simple scaling rule by second order approximation of the 
cross ambiguity function. For g and 7 being even and real, the squared cross ambiguity can 
be approximated for small |//| as follows (see Appendix [A)) 

\Ag^ifi)\'^{g,-f)'[l-A7r\fila^ + filaj)] (22) 

with = {t^g,i) / {g,i) and a'j = {Pg,^) / {g,i)- The latter is a slight extension of the 
often used approximation for the auto-ambiguity function {g = 7), which gives ellipses as 
contour lines of (|22l) in the time-frequency plane [26]. The optimization problem for the 
averaged channel gain turns now into the following scaling problem 

max / |Ag^(/i)pC(/i)(j/i ^^'^^ '^°"'^'> min / \jji\a1 + ^\aV\C{^)dii (23) 

(7,ff) Jk2 {Tt,Tf) Jk2 
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which is an optimization of at and o"/ only. For a separable scattering function C(/i) = 
Ct{fii)C f{ii2) this further simplifies to 

min C'-^^a^ + C^'^aj (24) 

where C^^^ = \\Ct\\i J Cf{uydu and C^*) = ||C/||i/Ct {T)T'^dT are the corresponding 
scaled second moments of the scattering function. Optimal points have to fulfill the relation 



— ^[C(^)a,2 + C^%)] =0^^ = J^ (25) 



Already in [14] this matching rule was found for C{fi) = 2Bprd ^[-^,^] (/^2) 
(non-causal) and for centered flat elliptic shapes, which can easily verified from (1251) to 
be (Jt/af = Td/{2Bj:,). For this special case the rule was also studied in [27] whereas 
the latter derivation of the scaling law in terms of moments needs no further assumptions. 
The pulses itself which have to be scaled accordingly were not provided by this second 
order approximation. But in [25] we have shown that an operator-algebraic formulation 
leads to eigenvalue problem for Hermite-kind differential operator, having for small C^^^C^^^ 
Gaussians as optimal eigenfunctions. Moreover this approach gives some hint on the optimal 
phase space displacement between g and 7, which is also not provided by (|22|) . Hence, what 
follows is, that Gaussians are a good choice for underspread channels (C^^^C^^^ <^ 1) if only 
pulse scaling is considered. 

2) Step two (Interference minimization): The main objective in this step is the minimization 
of the upper bound on the sum of interferences from other lattice points, i.e. E->^{6} < B^ — 
E-K{a}. Let us consider a pulse pair {g, 7} that is returned by step one. Hence they represent 
some kind of "single-pulse channel optimality" achieved by scaling or direct solution of the 

def 

gain optimization problem, i.e. let us say they achieve the value F{g,j) = E->^{a}. As we 
will show later on in more detail the optimal value of F{g, 7) depends only 7, where g is given 
by the "optimum", i.e. -^(7) == max||g||2=i F{g, 7) and SINR(7, A) = max||g||2=i SINR(5(, 7, A). 
Recalling now the Bessel bound = p{Sy^\), we introduce a linear transformation 7 — > Q'j 
in (fT4)) such that 

min SINR(7, A)"^ < min (a^ + p(5q^,a)) /F{Q^) - 1 (26) 

7 Q,7 

To arrive at what is commonly known as pulse orthogonalization, we have to ensure that 
F(Q'j) ^ -^(7) which is correct for non-dispersive channels {F(Q'y) = -^(7) = 1). But for 
the doubly-dispersive case this could be different and it is exactly the gap which can be filled 
by non-orthogonal pulses. Nevertheless, under this assumption we would have 

minSINR(7, A)"^ < min (cr^ + p{Sqj a)) /maxF(7) - 1 (27) 

7 Q '7 
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where the maximization is solved already by step one. The remaining minimization can be 
performed independently of 7 as follows. If Q = (38^ and if ^(7,A,Z^) establishes a 
frame, we have 



Sq,a = {SxQi, ■)SxQi = QS,,aQ* = = ^ (28) 

AeAZ2 

because commute with each Sx for A G AZ^ so its powers. Thus, with a ^ 
we obtain a tight frame which has minimal Bessel bound = |detA|, i.e. we achieved 
equality in (|27|) . This well known procedure [10] was already applied for the pulse shaping 
problem in [8] and has it origins in frame theory. Independently a different method for 7 
being a Gaussian was proposed in the context of OQAM [3] which yields the so called 
IOTA pulse (IOTA= Isotropic Orthogonal Transform Algorithm). It is known that IOTA is an 
equivalent method to obtain a tight frame [28]. But note that this method does not work in the 
general case. Furthermore because of the integer oversampling (two is needed for OQAM) 
the calculation of S*"^ simplifies much in the Zak-domain and can be done using efficient 
FFT-based methods [29]. The extension to the case where ^(7, A, I?) is an incomplete Riesz 
basis is done by Ron-Shen duality. In this case the minimal Bessel bound is achieved by an 
ONB, which is given by ^(5*^107, A, Z^), i.e. given by the computation of a tight frame on 
the adjoint lattice. Interestingly the resulting orthogonalization procedure based on duality is 
equivalent to the known Schweinler-Wigner [30] or Lowdin [31] orthogonalization. Hence, 
we arrive at the following operator 

(s'l |detA|<l 
0,,A = Pl '\ (29) 
y S^J^o else 

to be applied on 7 to minimize the Bessel bound {(3 ensures the normalization). To perform 
this operation a lattice with det A"^ = e has to be fixed. In the view of our previous derivations 
it would be desirable to choose the A which minimizes 5i(A) = |F(0^,a7) ~ -^(7)1 which 
is a rather complicated optimization. However, it is known that a7 is closest to 7 in the 
^2-sense [32], i.e. min^^ \\d — 7II2 = ||0^^a7 — 7II2 = <^2(A). The relation between 5i(A) and 
52(A) is out of the scope of this paper, but it is 5i(A) whenever 52(A) — 0. 
Lattice scaling: For A = diag(r, F) and 7 being a Gaussian it is in principle known, that 
T/F = (Jt/(jf ensures the minA5i(A). Moreover, in terms of the channel coherence one can 
follow the argumentation given in [27], i.e. 

V^C^ < T < i and \^C^ <F< (30) 
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In summary, the overall scaling rule for the lattice and the pulse according to the channel 
statistics is: 

T/F = at/cyf = yJc^'^/C^f^ (31) 

B. Mathematical Formulation 

In the following we will investigate the mathematical structure of the problem more in 
detail. Observe that the squared magnitude of the cross ambiguity function |Ag^(/i)p can be 
written in the following form 

|A,,(/i)|2 = {g,S,^){^,Slg) = Tr GS.TS; (32) 

where G and T are the (rank-one) orthogonal projectors onto g and 7, i.e. Of = {g, f)g. 
The linear functional Tr(-) denotes the trace, i.e. let us define Ti as the set of trace class 

def 

operators. The set 6 = {2; | 2; G 7^, ^ > 0, Tr 2; = 1} is a convex subset of With Z we will 
denote the extremal boundary of G, which is the set of all orthogonal rank-one projectors. 
Now let us transform the averaged channel gain furthermore in the following way 

En{a}= [ Tr{GS^TS*^}C{i^)dn= Tr{G [ S^TS;C{fi)dfi}= TrGA{T) (33) 

where we have introduced the map A{-). It maps operators X as follows 

A{X) [ S,XS;C{fi)d^i (34) 

This integral is mean in the weak sense. With = ^JC{^)Sfj, the map A{-) can written in a 
standard form known as the Kraus representation A{X) =^ dfiK^j^XK*, which establishes 
a link to completely positive maps (CP-maps) [33]. CP-maps like A{-) received much attention 
due to its application in quantum information theory. They represent stochastic maps on the 
spectrum of X. Let us collect some properties: 

Ais unital <^ A{I) = I 

Ais, trace preserving TrA(X) = TrX 

(35) 

Ais hermiticity preserving A(X*) = A{Xy 

Ais entropy increasing A(X) ^ X 
where >- is in the finite case the partial order due to eigenvalue majorization. Recall now the 
gain maximization (step one in IIII-A.ll) . i.e. (l20l) written in the new formulation is 

max Tr GA(T) < 1 (36) 

G,r&z 
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For the non-dispersive and single-dispersive case (all ^/CjjTjS^ commute pairwise, otherwise 
we call the channel doubly-dispersive) it is straightforward to solve problem (l36l) and it turns 
out that the optimal value (the upper bound) is achieved. This does not hold anymore for the 
doubly-dispersive case. The formulation in (l36l) is closely related to the maximization of the 
quantum channel fidelity criterion in the area of quantum information theory. In fact - the 
problems are equivalent if considering so called pure states (the rank-one requirement). It 
is quite interesting that the CP-map A(-) considered here corresponds for a Gaussian C{fi) 
to the classical (bosonic) quantum channel (see [34], [35] and also [36]). In [36], [23] it is 
shown that in this case the maximum is achieved by so called coherent states (time-frequency 
shifted Gaussians). Furthermore, the important role of Gaussians as approximate maximizers 
is also assessed by the authors in [25]. The application of cyclic shifts in the trace functional 
gives rise to the definition of another map A{-) by 

Bn{a}=TrT[ S*GSf,C{fi)dfx= TrTA{G) (37) 

def 

which is the adjoint of A{-) with respect to {X,Y) = TrX*Y. Based on the Weyl- 
Heisenberg group rules, i.e. 5^ = e~''^^^''^^ S^^ , S^S^ = e~^'^^^'^'"' S and S^S^ = 
^-i2n{tJ,iU2-fi2Ui) ^ have, that CP-maps with Weyl-Heisenberg structure are covariant 
with respect to group members, i.e. 

A{S,TS;) = S,A{T)S; (38) 

Thus, (l36l) is invariant with respect to joint time-frequency shifts of T and G. A trivial 
but important conclusion is that Weyl-Heisenberg (Gabor) signaling is a reasonable scheme, 
which guarantees the same averaged performance on all lattice points. Another conclusion 
is that two CP-maps Ai(-) and A2{-) both having Weyl-Heisenberg structure commute, i.e. 
Ai o A2 = A2 o Ai. Finally, a CP-map A(-) with Weyl-Heisenberg structure is self-adjoint 
(A = A) with respect to the inner product {X, Y) = TrX*Y, whenever C(/i) = C{—fi). 
Similar transformations can be performed on the interference term, i.e. 

En{b}=TrG ^AA(r)51 = Tr(G ■ (5 o A)(r)) (39) 

AeAX\{0} 

The introduced map B 

BiT)"^ J2 s.rsi =J2Sxrsi-r ^^^^ 

\eM\{o} AeAX 
is also hermiticity preserving and fulfills B{T) -< T. For |X| < 00 follows B{T) E Ti 
whenever F G T^. Hence, with p =^ (|X| — 1)^^ follows pB is unital and trace preserving. 
Finally let us define 

G = BoA and D{X) = + G{X) = G{a^ + X) (41) 
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With this definitions the optimization problem reads now 

TrA{X)Y TrXA{Y) 

max ^ = max = (42) 

x,yez TrD{X)Y x,Y(iZ TrXD{Y) 

IV. Optimization Strategies and Algorithms 

In this part of the paper we will discuss the desired problems in view of possible opti- 
mization strategies and algorithms. Hence we will now consider as the underlying Hilbert 
space. The set of trace class operators are now represented by L x L matrices and the set 
6 are positive semidefinite matrices having normalized trace. The matrix representations of 
time-frequency shift operators are given as {S^j,)^^ = 5m,neMi^*^''^^ where all index- 
arithmetics are modulo L. Thus /i G Z| where Zl := {0, . . . , L — 1}. 

A. Convex Constrained Quasi-convex Maximization 

We focus now more in detail on the problem (|42|) . It is straightforward to see, that one of 
the optimization variables can be dropped, which is 

Fx^z Tr d[x]y = maxA„,,.(A(X),D(X)) = m^xX^,^{AiY), D{Y)) (43) 

If we drop Y it is left to maximize the generalized hermitian eigenvalue Amax(^(-^), D{X)) 
or if we drop X it is \„ii^^{A{Y), D{Y)). The maximal generalized hermitian eigenvalue 
Amax(^(-^)5 -D(X)) is a quasi convex function in X if D{X)~^ exists, i.e. all level sets are 
convex. The existence of the inverse is ensured by cr^ ^ 0. Independently it can be shown 
that for cr^ = this can also be achieved with | det A| < 1 and X = 1? . From the quasi 
convexity follows 

maxAmax(^W,^(^)) = maxA„,ax(^(VpiXi),Z}(VpiX,)) = 

i i 

maxAi^ax(y]piA(Xi), VD(piXi)) < max{max Amax(^(^i), ^(^i))} (44) 

i i 

= me.xX^^,{A{X),D{X)) 

Thus, the optimization can be performed over & but the set of maximizers contain at least one 
X G Z (the maximum is at least achieved at some vertex). Moreover under our assumptions 
{D{X)~^ exists) the generalized eigenvalue can be rewritten as the (classical) eigenvalue 
maximization problem maxx^z ^ma.x{A{X)D{X)~^). But note that the argument is now 
rational in X. However, we aim at maximization of a quasi convex function over a convex set, 
which is a global optimization problem (commonly formulated as a minimization problem, 
i.e. quasi-concave minimization). As we will expect that the dimension of the Hilbert space 
will be large , i.e. L ^ 1024 . . . 8192, standard techniques for global optimization are mainly 
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prohibit. Typically branch-and-bound algorithms are able to find a global optimum for low 
scale problems. But for the lowest possible dimension L = 2 the problem can be solved 
completely [37]. For our setup we will instead propose in the following a simple algorithm 
which provides a lower bound. Hence, we make the following mappings explicit 

y{X) = arg max SINR(X, Y) 

x{Y) = arg max SINR(X, Y) (45) 

z{X)'^{xoy){X) 

where SINR(X, F) = Tr A{X)Y / Tr D(X)Y. This single-variable maximizations can be 
efficiently solved by computing the generalized hermitian eigenvalues. Hence, y{X) (or 
x{Y)) is the generalized maximizing eigenvector of {A{X), D{X)} (or {A(Y), D{Y)}). 
The mapping z corresponds to one iteration step. The iterative algorithm is given below. 
Convergence in the weak sense is given straightforward by observing that {sinr(X„, F„)}„ 

Algorithm 1 SINR optimization 
Require: 6 > 

Require: an appropriate initialized state Xq (for example a Gaussian) 
1: repeat 

2: Calculate in the n'th iteration : 

x/=l'^"(Xo)=z(^("-^)(Xo)) 



Yn = yiXr.) 

3: giving functional values SINR(X„, F„) = SINR(X„, y(X„)). 

4: until SINR(X„,r„) - SINR(X„_i,y„_i) < 5 



(46) 



is monotone increasing and bounded. 

B. Convex Constraint Convex Maximization 

The following suboptimal strategy to the solution of the problem is very important and 
related to step one in IIII-A.1[ It can also be obtained by considering the noise dominated 
scenario (cr^ oo) 

maxA^ax(A(X)D(X)-i) = ^ max A.,ax(A(X)(l + \c{X))-') 

x&z (j^ x^z (J (47) 

— ^ ^maxAmax(^(-^)) for ^ oo 

As in (|44)) there holds the convex relaxation, because Amax(-) is a convex function in its 
arguments and A[-) is linear. Thus we can instead solve the following convex constraint 
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convex maximization problem 

maxAmax(^(X)) = maxAmax(i(l')) (48) 
xes Yee 

Again this type of "max — max" optimization is of global kind, hence methods depend 

strongly on the structure of problem. An iterative algorithm but much less computational 

costly as the iterative maximization of SINR is given with 

y{X) = argmaxF(X, y) 

x{Y) = arg max F{X, Y) (49) 
z{X)'^{xoy){X) 

def ~ 

where F{X,Y) = Tr A{X)Y = Tr XA{Y). The maximizations can be solved efficiently 
by eigenvalues decompositions, i.e. y{X) and x(Y) are the maximizing eigenvectors of A(X) 
and A{Y). The iterative algorithm is given below. The proof of weak convergence is again 

Algorithm 2 GAIN optimization 
Require: 6 > 

Require: an appropriate initialized state X° 
1: repeat 

2: Calculate in the n'th iteration : 

Xr, = z''{Xo)=z{z^''-'\Xo)) 
Yn = y{Xn) 

3: giving functional values F(X„, F„) = 
4: until - F(X„_i,F„_i) < 5 



straightforward. It has been turned out that this algorithms are extension of the so called 
"mountain climbing" algorithm proposed by Konno [38] for bihnear programming. This 
can be seen if considering a corresponding basis representation. It is known that the set of 
hermitian matrices establish a real vector space. Let {ui} be a basis, i.e. we have X = ^ . XjCTj 
and Y = J^. xjaj 

max 'TrA(X)Y = max (x,ay) (51) 

where the x = {. . . , Xi, . . .)'^ and y = (. . . , t/j, . . . )^ and a is a matrix with elements 
ttij = Tr A(ai)aj. The optimization problem looks now rather simple but the difficulties 
are hidden in the definition of the set = {x\ J^i^i'^i ^ ®} (^^^ Bloch manifolds [39]). 
Without going further in detail we can already state, that if A{-) is self-adjoint (A = A, i.e. 



(50) 
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for C(yu) = C{—^)) the matrix a is symmetric. Then the bilinear programming problem is 
equivalent to convex quadratic maximization [38] 

max (x, ax) (52) 

Finally let us point out that separate interference minimization is formulated in this framework 
as 

min TyC{X)Y = mmK,^{C{X)) = min An.in(C'(F)) (53) 

x,Yez xez Yez 

which was already studied in [40] by means of convex methods. Unfortunately also this 
problem itself is not convex. It is again concave minimization over convex sets, because the 
convex relaxation (Z &) applies here as well. For non-dispersive and single-dispersive 
channels in turn this can be shown to be equivalent to the Bessel bound minimization (IIII-A.2I) . 

V. Performance Evaluation 

In this section we will evaluate the performance of the proposed pulse shaping algorithms. 
We compare them to the performance obtained by the use of properly scaled Gaussians, 
IOTA function and rectangular pulses. 

A. WSSUS Grid Matching and Pulse Scaling 

We use a rectangular lattice A = diag(T, F) properly scaled to the WSSUS statistics. 
Remember that pulse and lattice scaling with respect to a causal "flat" scattering function 
with support [0, r^] x [—Bo, B^] in the discrete representation means to fulfill approximately 

- ^ ~ I with (TF)"^ = e (54) 



2Bd (Jf F 

according to equation (|3TI) . Under a fixed bandwidth constraints W and fixed bandwidth 
efficiency e (in complex symbols) this lattice scaling rule (grid matching) can be easily 
transformed into an optimal number of subcarriers. Given TF = e~^, T/F = T(i/(2Bd) and 
F = W/N, where is the number of subcarriers, follows 



'^-'^V^-'V^ '''' 

where v is the speed between transmitter and receiver, c the speed of light and fc the carrier 
frequency. Moreover, in FFT based polyphase filtering A^ has be a power of two. The rule (l55l) 
represents nothing more than the tradeoff between time and frequency division multiplexing 
in time-variant channels. 
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B. Performance of the Iterative Algorithms 

We have verified our iterative optimization algorithms for a complex scheme (at e = 
0.5) and a real scheme (e = 2 and OQAM) on pulses of length L = 512. Assuming 
then a bandwidth W discrete time-frequency shifts jj,^^^ E Z| are related to M? 3 jj = 
diag(l/W,l^/L) ■/i(^), i.e. 

M"V/^f = T-^ = ^ (56) 

If we let B^^^ and r|^^ be the discrete maximal Doppler shift and delay spread, the support 
of the scattering function is then [0 . . . rj^'')] x [—B^^'' . . . -B^''] of fixed size P := (rj^^ + 
l)(25jf ^ + 1). Only the discrete ratio: 

R ■■= (rf^ + l)/i^B^D^ + 1) (57) 
has been varied according to the following table: 








1 


5 


9 


29 


49 


149 




74 


37 


12 


7 


2 


1 





N{e = 0.5) 


1 


2 


8 


16 


32 


64 


256 



This gives P ^ 150 for all R, thus with (l56l) follows P/L ^ 0.29 which is a rather strong 
but still underspread channel. Furthermore the number of subcarriers N is matched to R 
according to (1551) as much as possible, but such that still e = 0.5 and L mod = 0. For 
OQAM (e = 2) the grid matching has been repeated with the result that the number of 
subcarriers has to be simply 2A^ to fulfill the requirements. Where the minimal and maximal 
values of R (first and last column in the previous table) correspond to single-dispersive 
channels (either time-variant, non-frequency-selective or time-invariant, frequency- selective) 
the value in between are fully doubly-dispersive. Furthermore a noise power of cr^ = — 20dB 
is assumed. The SINR-optimal timing-offset between g and 7 for the Gaussians, IOTA and 
rectangular pulses is verified to be consistent with our theoretical result [25]. 

1) Complex Scheme at e = 0.5: The design criterion here is robustness not bandwidth 
efficiency. In FigH] we have shown the obtained averaged channel gain E7^{a} for the 
iterative algorithms in comparison to Gaussian pulses, IOTA pulses and rectangular pulses. 
Furthermore the result of lower bound (|2TI) is included, which achieves the same value as the 
iterative gain optimization. Remember that IOTA and the tighten version of the iterative result 
("gaintight") represent orthogonal signaling (via (|29|)). The orthogonalization does not really 
change the optimality of the gain optimal solution but significantly reduces the interference 
as shown in Figl2l Finally this increases the SINR as shown in Figl3l But the maximal value 
is achieved directly with iterative SINR optimization which yields a non-orthogonal signaling. 
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Fig. 1. Ew {o} = F{g, 7) for a "flat" underspread WSSUS channel (2TdBD ~ 0.29j for a complex scheme (at e — 0.5) 
- The achieved channel gain is shown for the results from the "SINR optimization", "gain optimization", its tighten version 
via J29t . The lower bound ( 121b achieves the same value as the pulses obtained from the "gain optimization". Furthermore 
TF-matched Gaussians, TF-matched IOTA and TF-matched rectangular are included. Note that: "SINR optimal" is not "gain 
optimal". 




Fig. 2. E->i{6} for a "flat" underspread WSSUS channel (2TdBD ~ 0.29j for a complex scheme (at e = 0.5) - The 
averaged interference power is shown for the results from the "SINR optimization", "gain optimization", its tighten version 
via M9\ . The lower bound ( 121b achieves the same value as the pulses obtained from the "gain optimization". Furthermore 
TF-matched Gaussians, TF-matched IOTA and TF-matched rectangular pulses are shown. The minimum is achieved with 
non— orthogonal pulses. 



Febraaiy 7, 2008 



DRAFT 




Fig. 3. SINR((;,7) /or a "flat" underspread WSSUS channel (2TdBD ~ 0.29) /or a complex scheme (at e — 0.5) - The 
SINR is shown for the results from the "SINR optimization", "gain optimization", its tighten version via l |29l l. The lower 
bound l l21t achieves the same value as the pulses obtained from the "gain optimization". Furthermore TF-matched Gaussians, 
TF-matched IOTA and TF-matched rectangular pulses are shown. The maximum is achieved with non-orthogonal pulses. 



2) Real Scheme (OQAM): The gain optimization does not change for OQAM, hence is the 
same as for the complex scheme. The difference is in the interference term. OQAM operates 
a maximum spectral efficiency which can be achieved for linear-independent wave functions 
with respect to real inner products. Hence there is no redundancy in the expansion. From (fT9l) 
it is to expect that both iterative optimization algorithms must yield similar results, whenever 
the result of the gain optimization establishes a nearly tight (snug) frame, i.e. if B^/A^^ ^ 1. 
This is indeed the case as shown in Fig|4l The SINR-performance gain with respect to the 
scaling approach based on Gaussians (IOTA) is about 3dB. 

3) Implementation Notes: Note that it is still a numerically challenging task to apply the 
iterative pulse design algorithms on real scenarios. For that one has to optimize for L ^ AN 
and more, where the number of subcarriers has to be according to (l55l) . Considering 
practical scenarios, for example e = 2 (OQAM), = 2GHz and W = 7.68MHz, it turns 
out via ([55]) that N = (256, )512, 1025, 2048(, 4096) are feasible [5], [41]. Optimizations up 
to L = 1024 are in principle possible with conventional direct implementation (using C and 
Matlab and no FFT processing). For L > 1024 only the "Gain optimization" and the lower 
bound remains. The convergence of the "Gain optimization" is rather fast, we achieved the 
values after 2 ... 3 iterations. For the "SINR optimization" which is the most computational 
expensive task the convergence is slower. We stopped the algorithm after 5 iterations. 
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Fig. 4. SINR(,g,7) for a "flat" underspread WSSUS channel (2TdBD ~ 0.29) for a real scheme (at e = 2 and OQAM) 
- The SINR is shown for the results from the "SINR optimization", "gain optimization", its tighten version via l l29t . All 
algorithms and the lower bound | |2U achieve mainly the same values. Furthermore TF-matched Gaussians, TF-matched 
IOTA and TF-matched rectangular pulses are shown. 



VI. Conclusions 

We have shown, that pulse shaping with respect to the second order statistics of a WSSUS 
channel is challenging optimization problem. We have introduced a new theoretical framework 
which straightforwardly yields two abstract optimization problems which can be partially 
related to other areas of quantum physics and mathematics. Unfortunately, due to non- 
convexity of these large scale optimization problems no standard methods applies. In fact, 
global solutions will strongly rely on the structure and can be obtained only for special 
cases. But with the presented iterative algorithms we have verified that even for an advanced 
multicarrier transmission like OQAM/IOTA potential improvement (3— 6dB in SINR for strong 
doubly-dispersive channels) can be expected. Moreover, it is very likely that additional gain 
can be obtained if also advanced receiver structures instead of one-tap equalizers are used. 
Those will profit much more from the sparsity of the effective channel decomposition. 

Appendix 

A. The o{2)- approximation for cross ambiguity functions 

In this part we will provide a slight variation of a well known approximation for auto 
ambiguity functions, in the context of radar theory probably first time presented in [26]. 
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Recall that the cross ambiguity function of g and 7 is given as 

A,^(/i) = {g, 5^7) = j me^'^'^'Mt - l^i)dt (58) 

If we plug in the following series expansions 

7(t - /ii) = 7(t) - /ii7(t) + ]^^^\l{t) + 0(2) and e*^"^^* = 1 + 227r/i2t - 271^ ^il^ + o(2) 

(59) 

we get the following approximation for the cross ambiguity function 

(60) 

271^ {filig, e-i) + ^i\{g, f^)) + z27r/ii^2((7, ^7) + o{2) 
where t, f are multiplication operators (with the variables t in the time domain and with / 
in the Fourier domain). The functions 7 and g denote the Fourier transforms of 7 and g. Let 
us furthermore assume that g ■ 7 and ^ ■ 7 are symmetric. This is fulfilled if for example g 
and 7 are real and itself symmetric. Then we have 

= {9, 7) ■ (1 - 27r2(/,2^2 ^ ^2^2) ^ ,27r/xi/i2^^^ + 0(2) (61) 

where af = (^^f?, 7)/(fi', 7) and aj = {Pg,^) /{g,^)- For the squared magnitude of A^^ we 
find 

|A,»P = |(^?,7)r (l-23?(27r2(/iX2 + ^2^J)-z2w2^^)) + o(2) (62) 
Finally let us force g and 7 to be real, which gives the desired result 

\AM\' = {g, 1? ■ (1 - An\^^lal + ^^la))) + o(2) (63) 
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